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Digital camera having CMOS image sensor with improved dynamics and 
method for controlling a CMOS image sensor 

This invention relates to a digital camera having a CMOS image sensor with a 
plurality of pixels each storing within an exposure time a brightness-dependent charge 
which is outputted as a pixel signal under the control of a control device, a correction 
device which, after a pixel RESET of the image sensor, forms the difference from first 
and second sampled values detected toward the onset and end of the exposure time for 
the pixels, and outputs the second sampled value reduced by the first sampled value as 
the wanted signal. 

The invention also relates to a method for controlling a CMOS image sensor. 

For digital cameras and the like for example, CCD image sensors have been 
known for some time. There are also CMOS image sensors, which have poorer bright- 
ness dynamdcs compared to CCD image sensors. The term 'Tjrightiiess dynamics" re- 
fers to the ratio between saturation signal amplitude of the exposed image sensor and 
its noise signal amplitude when the sensor is unexposed. Unless otherwise stated here, 
all remarks relate to any pixel of the image sensor wherein a plurality of pixels are dis- 
posed in rows and columns. 

With the image sensors in question here, a certain quantity of electricity is col- 
lected within a certain exposure time in each pixel of the image sensor. The charge is 
converted to a voltage with the aid of a capacitor. The quantity of electricity that can 
be taken up by a pixel has an upper limit. With a predetermined exposure time there is 
a certain brightness at which the collected quantity of electricity just reaches the upper 
limit. At greater brightness the upper limit for the collectable quantity of electricity is 
already reached earlier, so that the level of the wanted signal obtained at the end of the 
exposure time no longer says anything about the quantity of light taken up. 

There are several noise sources for an unexposed sensor. Besides (thermal) sys- 
tem noise and dark current noise, there is so-called RESET noise, also known as kTC 
noise. At the onset of an exposure time a RESET signal is applied to the pixels to drain 
charges stored in the pixel up to then. But the relevant pixel also delivers a signal di- 
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directly after a RESET process, namely the noise signal. This kTC noise constitutes 
the predominant noise component. With CMOS image sensors there is additionally the 
switch-over noise of the column multiplexer. When the sensor is exposed there is also 
photon noise. 

In addition to the stated noise signals, a CMOS image sensor has a time-constant 
noise source. It has the form of dark value offsets. This noise is frequently also Icnown 
by the rather incorrect name of "fixed pattern noise"; it does not involve a fluctuation 
noise source but a time-constant noise source. This noise varies from pixel to pixel and 
can be taken into account in simple fashion, as explained in more detail below. 

In order to suppress falsification of the wanted signal in image sensors by RE- 
SET noise, it is known to correct the signal sampled value obtained at the end of the 
exposure time by subfracting a noise sampled value from said sampled value. For this 
purpose, a first sampled value corresponding primarily to the RESET noise is obtedned 
directly after the RESET signal. The first sampled value is then subtracted from the 
second sampled value obtained at the end of the exposure time. Since both sampled 
values have the same RESET noise, the differential signal is substantially free from the 
RESET noise. 

For the prior art, reference is made for example to U.S. patents nos. 5,742,047; 
6,1 1 1,242; 6,115,066 and 5,877,715. The above-described method of forming a differ- 
ence for correcting the wanted signal is also described for CMOS image sensors 
therein. The subtraction can be performed using analog signals, but it is also possible 
to process the signals digitally, in particular in CMOS image sensors since these sen- 
sors have a separate charge-to-voltage converter capacitor for each pixel. It is then ex- 
pedient to have an image memory for storing the total dark image information for all 
pixels in digital form. The stored dark image digitized with the aid of an analog-to- 
digital converter is then subtracted pixel by pixel from the digital image in accordance 
with the sampling performed at the end of the exposure time. 

The abovementioned method with correction by subtraction of two sampled val- 
ues is known from the prior art, being referred to therein as correlated double sampling 
(CDS). The stated prior art already discloses said CDS method for CMOS image sen- 
sors. However, the effort necessary for the digital CDS method is so great that it is 
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that it is usually dispensed with. For this reason, imaging systems with CMOS sensors 
have the reputation of low dynamics. 

The invention is based on the problem of providing a digital camera of the 
abovementioned kind which has higher brightness dynamics compared to the prior art. 

A method is also to be provided that creates a precondition for improving the dy- 
namics of a CMOS image sensor. 

This is obtained in a digital camera of the abovementioned type by the control 
device delaying the sampling for the first sampled value by a predetermined delay time 
after the RESET. 

The invention is based on some basic findings which shall be discussed in more 
detail in the following. 

Firstly, in the prior art it is assumed that the first sampled value obtained is a 
mere noise signal containing no image information whatsoever, and subtracting this 
pure noise signal fi:om the second sampled value then substantially yields the pure im- 
age information. The noise signal caused by kTC noise is a random signal which has 
an amplitude firozen during the RESET process and remaining unchanged throughout 
the following exposure time. Accordingly, a first sampled signal obtained with a delay 
after the RESET thus also contains precisely the noise signal component caused by 
kTC noise and corresponding to the noise signal obtained at RESET time. Subtracting 
the first sampled signal - obtained with a delay after RESET according to the inven- 
tion - firom the second sampled signal thus likewise yields a wanted signal firee from 
kTC noise. 

The image component contained in the first sampled signal obtained with a delay 
gets lost in the subtraction process, but this loss is negligible in view of the multiplied 

brightness dynamics, as explained in more detail below. 

Secondly, as mentioned above, when the image sensor is exposed there is photon 
noise as well as the various other noise components. At high brightness, in particular 
brightnesses at which the CMOS image sensor tends to saturate, the photon noise 
component is so great that it overshadows kTC noise, which is independent of bright- 
ness. For this reason, kTC noise plays a minor part in image signals caused by high 
brightness. 
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Subtracting the first sampled value from the second sampled value obtained at 
the end of the exposure time yields a signal jfree from RESET noise, but this signal is 
reduced somewhat, i.e. by the image information component, in comparison to the 
prior art, where the first sampled value is detected directly after RESET. If, as pro- 
vided in a preferred embodiment of the invention, the delay time is selected so as to 
correspond to about 1 percent to 20 percent, preferably 2 percent to 10 percent, of the 
total exposure time, the amplitude of the wanted signal is reduced due to the subtrac- 
tion by about this percentage in normal operation. 

In exchange, however, a considerable advantage is obtained, namely greater 
brightness dynamics of the image sensor. When a pixel saturates, i.e. reaches its upper 
possible charge limit, very fast at high brightness, the wanted signal from said pixel is 
useless since the saturation signal contains practically no more information about how 
strong the exposure within the exposure time was. 

According to the invention, the first sampled signal can be used as the informa- 
tion source in such situations. The first sampled signal still has RESET noise - which 
is negligible at high brightnesses - but it also contains an information signal compo- 
nent, the signal amplitude being falsified by the fact that the exposure time for the first 
sampled signal corresponds only to the delay time. Since both the original exposure 
time and the delay time are known, however, the first sampled signal can be multiplied 
by a corresponding factor to extrapolate the first sampled value to a value correspond- 
ing to a signal at frill original exposure time. 

The above considerations also show that the low signal amplitude of the first 
sampled value provides considerable latitude up to which said first sampled signal can 
be used as the information signal even at extremely high brightnesses. If the delay time 
corresponds for example to 10 percent of the actual exposure time, the brightness dy- 
namics of the image sensor can be improved by approximately a factor often, assum- 
ing the second sampled value corresponds to the saturation signal amplitude. The first 
sampled value would also reach the saturation limit only at such high brightness. 

To permit the first sampled value to be actually used, in a preferred embodiment 
of the invention, the second sampled value can be compared with a threshold value. If 
the sampled value is higher than the threshold value, the first sampled value can be 
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multiplied by a corresponding scaling factor (derived from the ratio of actual exposure 
time to delay time) and the resulting value used as the wanted signal. 

Alternatively, an expected value for the second sampled value can be determined 
from the first sampled value using the scaling factor. If said expected value is above a 
certain threshold value, the wanted signal can then be determined from the first sam- 
pled value in the above-described fashion. 

At low and medium brightnesses the inventive digital camera can thus work like 
a conventional camera, while at medium and higher brightnesses it works in a new 
operating mode wherein the wanted signal determined by the CDS method is replaced 
by a wanted signal derived only from the first sampled value. 

In practice, undesirable edges or steps might occur in the image, i.e. if there is a 
sudden fransition from one operating mode to the other. For this reason it is provided 
according to the invention that the transition between the two abovementioned modes 
takes place gradually or fluidly. Accordingly, the following is outputted as the wanted 
signal in accordance with the brightness derived from the first and/or second sampled 
value: 

a) in a low brightness range: the difference of the sampled values; 

b) in a high brightness range: the value derived solely from the first sam- 
pled value; and 

c) in a medium brightness range: a weighted mixed value obtained from the 
values according to a) and b). 

The abovementioned fixed pattern noise, which is actually a time-constant noise 
source and not fluctuation noise, is taken into account according to the invention by 
providing a dark value offset memory and subtracting from the first sampled value of a 
particular pixel an offset value belonging to said pixel and prestored in the dark value 
offset memory. The offset values for the pixels of the CMOS image sensor can be 
stored before the first use of the digital camera by taking a picture with the sensor un- 
exposed, said values then being subtracted from the first sampled values of corre- 
sponding pixels at each image output. Storage of the dark value offsets can also be re- 
peated from time to time to update the values. 



The invention also provides a method for controlhng a CMOS image sensor by 
the so-called CDS method wherein pixels of the image sensor are sampled toward the 
onset and end of an exposure time, and a wanted signal is obtained by subtracting the 
first sampled value from the second sampled value, it being provided according to the 
invention that the first sampled value is obtained with usable image information after a 
predetermined delay time after RESET. 

In the following an example of the invention will be explained in more detail 
with reference to the drawing, in which: 

Figure 1 shows a block diagram of a digital camera with associated electronics 
for performing a CDS correction, 

Figure 2 shows the voltage curve in accordance with the increased quantity of 
electricity in a pixel of an image sensor according to an embodiment of the invention, 
and 

Figure 3 shows a signal pattern similar to Figure 2 but for a prior art CMOS im- 
age sensor. 

Figure 1 shows electric circuit 1 of a digital camera having CMOS image sensor 
2 which is controlled by control device 8 and is upstream of analog-to-digital con- 
verter 4. Analog-to-digital converter 4 converts the analog signals from the output of 
CMOS image sensor 2 into digital values which are then stored in dark image memory 
10 under the control of control device 8. The stored values in dark image memory 10 
correspond to the digitized voltages for the individual pixels of CMOS sensor element 
2. Said voltages are obtained after a RESET signal by a first sampling, controlled by 
control device 8. 

The sampled values obtained and digitized upon a second sampling at the end of 
an exposure time are formed by subtracter 6 into a differential signal by subtracting the 
first sampled values stored in dark image memory 10 from the digital second sampled 
values pixel by pixel and digitally, again under the control of control device 8. The 
differential signal at the output of subfracter 6 is supplied to wanted signal output^ via 
a first input of switch 12. 

The first sampled values stored in dark image memory 10 are also supplied to 
multiplier 14 where the sampled values are multiplied by scaling factor F. 
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Before multiplication, each pixel value from dark image memory 100 is cor- 
rected with respect to the aforementioned fixed pattern noise. For this purpose, offset 
values varying for the individual pixels of CMOS image sensor 2 are stored in dark 
value offset memory 1 1 for each pixel of CMOS image sensor 2 before the first use of 
the digital camera. At each readout process for image memory 10 the values are also 
read out in synchronism from the dark value offset memory. The values from memo- 
ries 10 and 1 1 are subtracted by subtracter 13, and the difference value, i.e. the first 
sampled value free from fixed pattem noise from image memory 10, is supplied to 
multiplier 14 to be multiplied by scaling factor F. 

The multiplied first sampled values are supplied to a second input of switch 12. 
The position of switch 12 is selected by control device 8, in accordance with an ex- 
pected value for the second sampled value. Said expected value is calculated from the 
particular first sampled values by control device 8. If the amplitude of the particular 
first sampled value is so great that the second sampled value can be expected to be 
above saturation threshold value Th, control device 8 switches switch 12 to the lower 
position, so that at wanted signal output^ a wanted signal is outputted which is ob- 
tained by multiplying the first sampled value by factor F . 

Figure 2 shows a sampling process in detail. Before Figure 2 is dealt with, the 
prior art shall be explained with reference to Figure 3. Reference is made here in par- 
ticular to the abovementioned U.S. patents, which are regarded as incorporated by ref- 
erence here, in particular with respect to the correlated double sampling method (CDS 
method). 

According to Figure 3, a charge collects in one of the pixels and is converted into 
voltage Fby a capacitor not shown here. The higher the brightness to which the pixel 
in question is exposed, the steeper the rise in voltage. At a given time CMOS image 
sensor 2 according to Figure 1 is reset pixel by pixel by a RESET pulse outputted by 
control device 8. This drains all residual charges from the individual pixels. 

Due to the RESET pulse, voltage F sinks abruptiy according to Figure 3 to a cer- 
tain value which is designated kTC here and corresponds to the noise signal amplitude, 
mainly due to RESET noise. 

According to the continuous line in Figure 3, voltage F already sampled for the 
first time at time T\ now rises ftirther, in accordance with the brightness to which the 



pixel is exposed. After given exposure time B there is a second sampling of voltage V 
at time 72. The signal amplitude at time T2 likewise has noise signal amplitude kTC. 
By subtracting the first sampled value obtained at Tl from the second sampled value 
obtained at T2 the noise signal component is eliminated. 

As further shown in Figure 3 by a dash-dotted line, the voltage would already 
reach saturation limit Th after about two thirds of exposure time B at somewhat higher 
brightness on the pixel in question. A sampled value obtained at time 72 would then be 
useless. 

According to Figure 2, the pixel in question is likewise RESET by the RESET 
pulse so that the voltage decreases to a noise signal amplitude, likewise designated 
kTC in Figure 2. 

In contrast to the prior art, however, the first sampling does not take place at or 
directly after the RESET pulse but at time TV delayed by predetermined delay time D. 
At this time the exposure of the pixel has already caused a certain quantity of electric- 
ity to collect, corresponding to a certain voltage, designated "Info" in Figure 2 to indi- 
cate that the signal sampled at time 71' contains information about brightness. 

When correction by subtraction of the sampled values at 72 and 71' takes place 
according to Figure 2, the obtained differential signal is firee from noise signal compo- 
nent kTC but is somewhat lower compared to the case shown in Figure 3, i.e. by the 
value designated "Info" in Figure 2. 

Let us now consider the case that the brightness on the relevant pixel is so great 
that signal voltage F rises very steeply in accordance with the dash-dotted line. The 
second sampled signal obtained at time 72 would then be useless. According to the 
invention, solely the first sampled signal obtained at time 71' is used as the wanted 
signal. It is multiplied by scaling factor F, which has the value B/D for instance, in 
order to take account of the fact that the exposure time at time 71' is greatly reduced 
compared to the actual exposure time and this reduction must be compensated for. 

At output A in Figure 1 one thus obtains, depending on the brightness, either a 
differential signal formed by subtracting the first sampled value at 71' from the second 
sampled value at 72', or a signal obtained from only the first sampled value at TV. 

Figure 2 indicates that the brightness dynamics of the image sensor according to 
the invention is many times improved in comparison to the prior art; voltage K would 



have to rise extremely steeply to reach upper saturation value Th by time 71' of the 
first sampling. 

Switch 12 shown in Figure 1 permits only the two alternatives, "differential sig- 
nal," on the one hand, or "wanted signal derived from first sampled value," on the 
other hand. To avoid image edges or steps one can replace switch 12 according to Fig- 
ure 1 with a circuit permitting a fluid transition between the two stated modes. In a 
medium brightness range a weighted mean value can be formed from the two types of 
wanted signal, "differential signal" and "wanted signal derived from the first sampled 
value," the weight being greater in the proximity of the low brightness range for the 
"differential signal" than for the wanted signal derived from the first sampled value. In 
the proximity of the high brightness range the distribution of weight is then reversed. 
This results in a fluid fransition between the two modes. ' 



